1. Field of the Invention
The present invention is directed to data backup and recovery.
2. Description of the Related Art
A number of direct access storage device (DASD) subsystems are capable of performing “instant virtual copy” operations, also referred to as “fast replicate functions.” Instant virtual copy operations work by modifying metadata such as relationship tables or pointers to treat a source data object as both the original and copy. In response to a host's copy request, the storage subsystem immediately reports creation of the copy without having made any physical copy of the data. Only a “virtual” copy has been created, and the absence of an additional physical copy is completely unknown to the host.
Later, when the storage system receives updates to the original or copy, the updates are stored separately and cross-referenced to the updated data object only. At this point, the original and copy data objects begin to diverge. The initial benefit is that the instant virtual copy occurs almost instantaneously, completing much faster than a normal physical copy operation. This frees the host and storage subsystem to perform other tasks. The host or storage subsystem may even proceed to create an actual, physical copy of the original data object during background processing, or at another time.
Instant virtual copy has been an important development in modern disk subsystems, and a number of different techniques have surfaced. As one example, International Business Machines Corporation (IBM) has developed the “FLASH COPY” technique, as described in different publications including U.S. Pat. No. 6,611,901, issued on Aug. 26, 2003, having U.S. application Ser. No. 09/347,344, filed on Jul. 2, 1999 and entitled “Method, System, and Program for Maintaining Electronic Data as of a Point-In-Time.” A different fast replicate technique is the “SNAPSHOT” technique disclosed in U.S. Pat. No. 5,410,667 entitled “Data Record Copy System for a Disk Drive Array Data Storage Subsystem,” which issued on Apr. 25, 1995. The foregoing references are incorporated herein by reference in their entirety.
Instant virtual copy techniques have been developed, at least in part, to quickly create a duplicate copy of data without interrupting or slowing foreground processes. A natural extension of this function has been the creation of a physical “backup” copy of the source data, to aid in disaster recovery. Under one such technique, an operation such as “FLASH COPY” or “SNAPSHOT” operation is used to perform an instant virtual copy operation; this creates a virtual target volume identical to the source volume in all respects. Then, the virtual target volume is taken off-line (i.e., is not accessible), which may occur automatically as a result of the instant virtual copy operation or manually at the direction of a system administrator.
Normal instant virtual copy operations can involve tens of thousands of files. Some application programs, such as SAP R/3 applications, using a database system, such as DB2®, may have 30,000 or more datasets. A dataset is a named set of records stored or processed as a unit. R/3 applications are a set of integrated business applications from Systems, Application and Products (SAP) in data processing. R/3 applications use a client-server model. R/3 applications enable storage, retrieval, and analysis of data for business processes. DB2® refers to relational database management system (RDBMS) products from IBM.
To perform backups of each dataset, applications may stop processing or process in read-only mode. In read-only mode, the applications read data in datasets, but do not update the data. It takes many hours to perform the backups for 30,000 or more datasets. Thus, for these hours, the applications are unable to update datasets, which is very inefficient.
In some cases, customers backup datasets “while open” (i.e., while one or more database transactions is in process) and rely on forward recovery and backout logs to resynchronize the data after recovery. In particular, each time a record is updated as part of a transaction, the “before” version of the record is stored in the backout log, and the “after” version of the record is stored in the forward recovery (redo) log. For example, it may take five hours to backup 30,000 datasets, with 6,000 of these datasets being processed each hour. If the backup datasets are to be used for recovery, the backup datasets are recovered (e.g., copied to a source from a backup copy), and the forward recovery or backout logs are applied to the recovered backup datasets. The forward recovery and backout logs are used to undo or redo updates of the backup datasets to get the backup datasets to match up to datasets at a certain time. If a backup was taken at midnight (12:00 a.m.), it is possible to apply updates from the forward recovery log for five hours forward to get a copy that represents datasets at 5:00 a.m. If a backup was taken at 5:00 a.m., it is possible to apply five hours of updates from the backout log to get a copy that represents datasets at midnight. In some cases, it takes three hours to apply one hour of the updates in the logs. Therefore, it may take up to fifteen hours to forward or backout a recovered dataset by five hours.
In cases in which datasets that belong to applications (e.g., SAP applications) are interrelated, the applications' datasets need to be backed up at the same time in order to recover (i.e., restore) the entire application.
Database management systems write logs of changes to database records along with other metadata. Many database management systems (e.g., DB2®) allow backups to be taken even when the database records are being updated, and these are referred to as “fuzzy” state backups. The database management systems allow the fuzzy state backups because the database management systems can recover the data to bring it back to a consistent state using the forward recovery and backout logs. Although this technique does not result in application outage (i.e., the application is allowed to read or write database records), the time that it takes to retrieve the metadata and establish a “instant virtual copy” (or “virtual concurrent copy,” e.g., FLASH COPY) for thousands of datasets is tens of minutes or more. Long fuzzy state backups also result in unacceptably long recovery times.
Some recovery systems perform two backups. First, the recovery systems generate image copies of individual datasets. Second, the recovery systems take one or more volume dumps for large scale recovery.
Furthermore, in cases in which the virtual copies are stored on physical magnetic tapes, it can be difficult to locate a particular dataset on a backup volume on the magnetic tape. In some cases, the magnetic tapes are manually reviewed to locate particular datasets. This process, however, is consumptive of time and processing resources.
Thus, there is a need for more efficient backup of data and data recovery.